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Patrick S. Yoder 



DECLARATION OF VI.TAYKALYAN YELURI UNDER 37 C.F.R. § 1,131 



I, Vijaykalyan Yeluri, hereby declare as follows: 



1. I am a co-inventor of record for the invention disclosed and claimed in the 
above-identified patent application. 



2. I am currently employed by GE Healthcare, a division of General Electric 
Company, in the Imaging Solutions Group. My current job title is Lead Software Engineer. 



3. 



My home address is 1223 Bracebridge Court, Campbell, California 95008. 
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4. I, along with the other co-inventors of record, conceived of the subject 
matter disclosed and claimed in the above-referenced application prior to September 23, 
2003. This conception is evidenced by an invention disclosure document that was prepared 
by one or more of either myself or the co-inventors of record and submitted to GE in the 
ordinary course of business prior to September 23, 2003. A true and correct redacted copy 
of pages 1-7 of this disclosure document is attached hereto as Exhibit A. 

5. Prior to September 23, 2003, a telephonic interview was conducted between 
outside counsel and one or more of the inventors of record, including Mr. Prakash Mathew, 
in which the preparation of a patent application for the subject matter set forth in the 
disclosure document was discussed. A true and correct redacted copy of an e-mail 
communication acknowledging the telephonic interview is attached hereto as Exhibit B. 

6. In view of the telephonic interview (Exhibit B) and the above-referenced 
disclosure document (Exhibit A), I and the co-inventors of record conceived of the 
invention ultimately disclosed and claimed in the above-identified application and were in 
fact in possession of the invention prior to September 23, 2003. 

7. Prior to November 26, 2003, the filing date of the above-identified 
application, I reviewed a draft of the above-identified application and executed the 
Assignment and Declaration papers for the above-identified application. The executed 
Assignment and Declaration papers were returned to outside counsel on or before 
November 26, 2003. Constructive reduction to practice of the invention disclosed and 
claimed in the above-identified application occurred on November 26, 2003. 

8. Diligent efforts were made from prior to September 23, 2003 through the 
filing of the present patent application on November 26, 2003. 
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9, The subject matter disclosed and claimed in the above-referenced 
application was conceived while in the United States. 

I declare further that all statements made herein are of my own knowledge, are trite 
and that all statements made on information and belief are believed to be true, and further, 
that these statements were made with the knowledge that willful false statements and the 
like so made are punishable by fine or imprisonment, or both, under 18 U.S.C. § 1001 , and 
that such willful false statements may jeopardize the validity of the application, any patent 
issuing thereon, or any patent to which this verified statement is directed. 

Dated: W c » 30. j^l By: \, _ 

Vijaykalyan Yeluri 



EXHIBIT A 



QT Invention Disclosure Form 


Docket No.: 133276 


Mail to: PATENT OPERATION, W-710 








Date Received : JIIB^BH^BEES 





> Use as many pages in this word document as necessary. 

• You may attach additional materials to support this disclosure, for example, Tech Notes and Drawings. 
Such submitted materials must be referenced in this disclosure form. Each page of these materials 
must be dated, signed and witnessed in the same manner as this invention disclosure. 



INVENTION TITLE : Pnn 

please provide- ,i ii:h< in Eugihh as 
franca is. 




Method and system for de-identifying burned in patient data from radiological images 



PROBLEM/BACKGROUND : Describe ti:c problem jhat is solved by the invention. .tssume that the redder has a basic 
knowledge of you diagnostic imaging modalin, vu > luted technoloi ei 

Privacy laws within the medical practice are starting to be heavily enforced (if not already) and a number of regulatory laws 
mandate the restricted and appropriate disclosure of patient information in all medical records. Radiologists routinely need to, 
and have been transferring images to referring physicians and colleagues for consultation or teaching, as part of their 
workflow. Radiological images often contain patient identifying data (mainly text), which would allow a casual or informed 
observer to associate the image with an individual. Rather than change their established workflow, radiologists look to 
information system vendors such as GE to provide the tools to de-identify images of patient data to render them compliant to 
the regulatory law. 

Sometimes, the textual annotations are separate layers or tags within an imaging protocol such as DICOM. In such cases, 
suppressing the display of these tags alone, based on some coded set of rules would be a solution. This however is easy to do 
and intuitive. A bigger problem arises when the textual information identifying the patient is part of the image pixel set itself, 
i.e. it has been burned in. In such a case the de-identification problem for radiological images is not trivial and the solution of 
tin's would be a considerable benefit to the radiological community. 
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C[ GE Medical Systems Invention Disclosure Form 



FN 1 'ON I 'Km. niPilOV Describe how th nvnt n orl > 'i v it so! ?th >km f >ea bo\ 
As described earlier, the problem is that of automatically and conveniently de-identifying "burned in" patient information 
from radiological images. 

Lets clarify what this invention will address. . . 

The most common radiological image interchange method is using the DICOM protocol format. The image is stored and 
transported in the form of an electronic file. There could be patient identifying information in: 

A) The DICOM headers: This is a text portion of the file that encapsulates some metadata regarding the image. Not all 
information will be visible in the displayed image, but it still represents data that could potentially reveal the identity 
of the patient. The d e-identification method is programmatically coding which information tags will be deleted or 
l an image. I 



B) The DICOM overlay tags: These are also structured information that could bebift 
image. They capture annotations, mar kers to regions of interests etc. 



ips that are laid over the actual 



C) The image pixel data. This is the actual pixel data with diagnostic information content. For convenience and record- 
keeping, patient identifying information is often burned into the exposure. This kind of patient data is hard to 
programmatically remove beca use there is no structured way of removing this data . The invention addresses this 
burned in data de-identification.H^^^^^^g^^^^^^^^^^^^^^^^" 1 



The solution we propose will be the novel use of Optical Character Recognition (OCR) technology for the de-identification 
of radiological images. OCR technology itself is not new or novel and is quite a mature area. This is routinely and heavily 
used in the area of text scanning applications. The following is an excerpt from a casual search on the www.About.com 
website for the term Optical Character Recognition. 

OCR (Optical Character Recognition) is the process of turning a picture of words (such as a scan of 
a typed letter) into an editable document that you can open and use in your desktop publishing 
software, word processor, or other text editor. Today's OCR packages contain sophisticated support 
for multiple languages, PDF and HTML output, and format retention. 



Ihas Page Formatter to better retain formatting of pages and recognization of 
spreadsheets, tables, and other elements. Convert to and from PDF format. The International Voice 
Read Back reads your text aloud and now recognizes over 100 languages. 



INVENTORS (Print or Type Name Below) 


(Full Signature Below) 


GE 


NOT GE 


DATE 


* Prakash Mathew 




X 






Yaseen Samara 




X 






Vijaykalyan Yeluri 




X 






Denny Lau 




X 







= Primary Contact Inventor (to coordinate with Patent Evaluation Board and Preparing Attorney) 



■> \\I 1 NfcSsj , , , (Print or Type Name Below) 


(Full Signature Below) 


DATE 















GE Medical Systems Invention Disclosure Form 



Page 3 



It retains page layout, allows output as Web pages, retains color in photos or graphics, and is 
Windows 2000 certified. This is a low-cost OCR option. 



A powerfulH5u^urpnsinglyine^ alternative to some of the better known (and expensive) 
OCR packages. It reads 52 languages, preserves the page formatting, can scan color and dark 
backgrounds, has batch processing, and has expert tools for analyzing complicated layouts. 



4^ 1 _, 

From^K^^M^w^^^^^^^^SS^^KMi t ' ie pro version is recommended for home use but 
is more advanced than the Home edition. Supports over 100 languages, has built-in spell-checker, 
handles PDF and HTML and retains document formatting. An under $100US OCR option. 



The Business Edition includes new and improved PDF Output, supports 56 languages, retains 
formatting, and outputs to HTML format. Includes scheduled batch processing for busy offices. 



The workflow involved might proceed something like this. 

1) A radiologist opens a study on a PACS system. He scans through the exams and selects one or more images for 
sending to another institution, lets say for example, teaching purposes. HIPAA mandates that patient demographic 
information should not be disclosed. 

2) The radiologist now selects the "De-identify including burned text" option in the PACS system. 



3) The PACS system now readies the images for sending out. It first negotiates the format of data it needs to send the 
images in. If DICOM, it creates a copy of the image file, removes the patient demographic information and any 
overlays in the header. Tf any other image format like GIF or JPEG, it deals only with die pixel data in the file. 
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4) In either case, the pixel data is fed into the OCR software program. This can be done progressively, m sections of 
the image, comprising a strip about 1cm x 5cm or some such small subset. Each section is quickly scanned and the 
OCR software determines if text is included in these regions. If text is found then an approximate position ot the text 
character is returned. Instead of subsets of strips of the image, the entire image can be fed into the OCR software as 
well. 

5) The PACS system then creates an opaque mask and bums it into the pixels at the locations where text has been 
recognized. This intensity value of the opaque mask value can be pre programmed into the system. As long as the 
mask is a single intensity value, the character information underneath is blocked out. 

6) The modified image, with the masks burnt in and obscuring the text is then chosen as the image to be sent to the 
external system. The mask can be patterned or with a distinguishable border so as to clearly communicate to the 
viewer that information is missing from that region. 

fotlhelidfu^ oftwarefl charts" or finished dmvL they may be > I 

An example of a radiological image with patient identifying information is shown below: The patient name is visible as text, 
along with other textual annotations. 




After the steps outlined in the invention occur, the resulting image could be modified to look like one of the following: 
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or like 




In both cases, the textual information is overlaid out by a burned in mask, thereby anonymizing the identity of the patient 
involved. 

In a related embodiment of the invention, specific rules can be incorporated in the software to recognize and exclude certain 
useful text which do not identify a patient. Useful text is in the form of measurements or view descriptions such as the phrase 
"FRONT" on the original image. Using this form of intelligent de-identification we could produce images like the following 




In this case, helpful information is retained while the patient identifying information is selectively masked out. 
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Block Diagram indicating for automatic de-identification 





No de-identification 
required for strip 
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There may be some cases, which will no doubt challenge the capacity of the invention. It will be cases where 

1 . The text is formatted vertically instead of the usual left to right approach. 

To address this problem ,we would configure the OCR software to read in vertical directions as well. This can be 
configured in addition to the left to right recognition. The user can choose to configure this or not, but the invention 
allows for this case, 

2. Lines running through the text could obscure the text or it could have extremely poor quality. 
For example, 




U>\ A MAT, IS OF Tin- 1V\ KMiQV- Describe the l ,mf>t ■ ' he inn ti m b >th U ' h licai it m (t g tro ger, 
lew application, faster imaging, etc ' and bushes terms eg cost savings, product 'Jicienc) etc ) 

1 . Automatic: This de-identification is done without any manual intervention. 

2. Flexibility: The de-identification can be carried out at or in any subset of the image space, based on a set of pre- 
determined rules. 

3 . Depending on the OCR software package used, a number of language texts can be recognized and removed. 

4. Security: Regulatory and safety guidelines and rules requiring image de-identification of patient information will be 
satisfied. 
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From: Mathew, Prakash (MED) [IMCEAEX- 

_0=GE_OU=GEMSAM_CN=RECIPIENTS_CN=| 

Sent: B^^f 3^^* , 'k^ p -^ 

To: Pat Yoder 

Cc: Samara, Yaseen (GE Healthcare) 
Subject: RE: tcon to discuss 133276 and 133277 



Hi Pat, . 
Regarding docket application, 133276, here is an example of | 



It wa; a pies e talking to you yesterday... 
Thanks 
_prakash 

Original Message-- 

From: Pat Yoder [mailto:| 
Sent: | "~ ~" — 

To: Mathew, Prakash (MED); Samara, Yaseen (MED, GEMS-IT) 
Subject: RE: tcon to discuss 133276 and 133277 

Sounds great Prakash. I'll look over the disclosures and we can try to fit everything in the hour. If we need more 

! i pi ,1- r ' > - 1 

Talk to you then! 



Original Message-- 

From : Mathew, Prakash (MED) [mailto :| 
Sent: " H 'I l'i Hi 

To: 'Pat Yoder'; Samara, Yaseen (MED, GEMS-IT) 
Subject: RE: tcon to discuss 133276 and 133277 

How a bout 10:30-11:30 CST ? 
TCON BHH^H #672651 ° 

__prakash 

Original Message-- 

From: P at Yoder [mailto:| 
Sent: I 

To: Samara, Yaseen (MED, GEMS-IT); Mathew, Prakash (MED) 
Subject: RE: tcon to discuss 133276 and 133277 

We have a regular meeting from il:30 to about 1:00 on Thursdays. Can we work around that? 
iPra^ash, you picked my one window on this one.) 



Thanks. 



— Original Message 

From: Samara, Yaseen (MED, GEMS-IT ) [mailtoj 

Sent: HHHHHHUIHSi 

To: Mathew, Prakash (MED); 'Pat Yoder' 
Subject: RE: tcon to discuss 133276 and 133277 



Sounds great! Thursday morning then. 

Original Message 

From : Mathew, Prakash (MED) 

Sent : fBHflHfiSBKffiHS 

To: 'Pat Yoder'; Samara, Yaseen (MED, GEMS-IT) 
Subject: RE: tcon to discuss 133276 and 133277 



Pat, Yaseen, 

I apologize for nouesponding earlier... 
How does Thu ffffifflH rning sound? 
How about 

1 1:00 - 12:00 CST TCON ■IHhShH #672651 0 (I'm flexible on 

the time that morning) 



prakash 



Original Message 

From : Pat Yoder rmailto:! 
Sent: | ~ 

To: Mathew, Prakash (MED); Samara, Yaseen (MED, GEMS-IT) 
Subject: tcon to discuss 133276 and 133277 



Guys, 

Any thoughts on this? I'm in all week, but probably need a couple hours 
advance notice before a tcon. 



Thanks! 



Pat 



Guys, 

As Yaseen and I discussed last week, we will be preparing patent 
applications on the referenced disclosures. I just returned from an extended 
trip and did review the disclosures during that time. I think the best way to 
proceed is through a telephone conference during which we can walk 
through the disclosures and I can flesh out points as needed to move along 
to the drafting stage. 

I will be in all week working on various projects. Could you please just let 
me know what works for you guys and I'll plan around that. I think we may 
need as much as a couple of hours to get through the material. One or both 
of you may with to participate, and I understand that Mathew will probably 
be the "point person" for these applications. 

Thanks, and I look forward to hearing from you. 



Pat 



Patrick S. Yoder 

Fletcher, Yoder & Van Someren 

Attorneys at Law 

7915 FM 1960 West, Suite 330 



Houston, Texas 77070 



Cel: HH 

fax: IHHHi^HflHI 



CONFIDENTIALITY NOTE 

The documents accompanying this e-mail contains information from 
the law firm of Fletcher, Yoder and Van Someren which is confidential 
or privileged. The information is intended to be for the use of the 
individual or entity named in this transmission. If you are not the 
intended recipient, be aware that any disclosure, copying, distribution 
or use of the contents of this e-mailed information is prohibited. If you 
have received this e-mail in error, please notify us by telephone 
immediately so that we can arrange for the retrieval of the original 
documents at no cost to you. 



